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ABSTRACT 


Recognition of partially occluded objects has been an important issue to airport security because occlusion causes 
significant problems in identifying and locating objects during baggage inspection. Neural network approach is suitable for 
theproblems in the sense that the inherent parallelism of Neural Networks pursues many hypotheses in parallel resulting in 
high computation rates. Moreover, they provide a greater degree of robustness or tolerra than conventional colters. 
The annealed Hopfield network which is derived from the mean field annealmg(MFA) has been developed to g 
solutions of a non-linear system. In the study, it has been proven that the system temperature of MFA is equivalents he 
gain of sigmoid function of Hopfield network. In our early work, we developed the hybrid Hopfield network(HHN) on the 
purpose of fast and reliable matching[l]. However, HHN doesn’t guarantee global solutions and yields false matching under 
heavily occluded conditions because HHN is depending on initial states by its nature. In this paper, we present the annealed 
Hopfield network(AHN) for occluded object matching problems. In AHN, the mean field theory is applied to the hy n 
Hopfield network in order to improve computational complexity of the annealed Hopfield network and provide reliable 
matching under heavily occluded conditions. AHN is slower than HHN. However, AHN provides near global solutions 
without initial restrictions and provides less false matching than HHN. In conclusion, a new algorithm based upon a Neural 
Network approach was developed to demonstrate the feasibility of the automated inspection of threat objects from X-ray 
images. The robustness of the algorithm is proved by identifying occluded target objects with large tolerance of their features. 


1. INTRODUCTION 


Pattern recognition and computer vision theory has been considerably improved during the last decade such that the 
appearance of an automated vision system seems very close to our future[2,3]. However, because of the higher computational 
burden of image understanding algorithms, use of object recognition from an image is still limited to the restricted 
environment. In the mean time researchers developed a new idea of computation which imitates human brain structure, called 
a neural computing^, 5]. Techniques in neural computing are based on a new concept of distributed parallel computation, and 
applicable to any number crunching objectives. Hopfield network, one of the neural computations is very popular in real world 
application due to simple architecture and well defined time domain behavior[5-9]. The Hopfield network is composed of 
single-layer neurons with fully connected feedback connections. The neurons have the sigmoid gain characteristic while the 
connectivity matrix corresponding to the connection is symmetric and the diagonal terms of the matrix are zero. Such networks 
always move in the direction of decreasing the energy of the networks and get stable states at the local minimum of the 
energy Since the energy function of a Hopfield network has many local minima, the resultant network output is usually the 
closest local minimum to initial states. This nature of the Hopfield network must be the dement in solving an optimization 
problem In our early work, we developed the hybrid Hopfield network algorithm to improve deficiency of the original 
Hopfield network. The method yields a good solution by adding an adjusting procedure for the output neuron states of the 
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Hopfield network. However, the method still does not guarantee global solutions. Simulated annealing is one heuristic 
technique to help escape the local minima by perturbing the energy function with the annealing temperature and artificial 
noise[10]. It is proven that the solution obtained by the simulated annealing is independent of the initial condition of the 
network and is usually very close to the global minimum[9]. Since the network should settle down at each temperature and 
the temperature decrement is very small, extraordinarily long time is required in the software computation. The mean field 
theory(MF) has been applied to the simulated annealing in the effort on reducing the computational time and many impressive 
outputs in image processing area have been reported. The MF has the analogy to the Hopfield networks[l 1]. It was proven 
that the system temperature of MF is equivalent to the neural gain. D. E. Van den Bout et ah also developed a new algorithm, 
the mean field anneal in g(M FA) which merges many features of simulated annealing and Hopfield networks[12]. They does 
not use a sigmoid function but use the normalization technique. However the normalization technique can be applied to the 
case in which the sum of a normalized subgroup is equal to 1. An occluded object matching problem is the one which can be 
cast into an optimization problem when the graph theory is applied to the problem. MFA can be used to solve the occluded 
object matching problem. Unfortunately, the problem is not suitable to normalization technique because the sum of normalized 
subgroups is zero or one. Thus, we use the sigmoid function, which is one of the important characteristic of Hopfield 
network. We can do an hardware implementation as well as an algorithm approach. We call this technique as the annealed 
Hopfield network(AHN). 


2. ANNEALED HOPFIELD NETWORK 


2.1 Feature extraction and graph formation 


In boundary based approaches, comer points are important since the information of the shape is concentrated at the 
points having high curvatures. From the comer points, we can extract useful features such as a local feature of an angle 
between neighboring comers and relational features of distances between the comers. These two features which are invariant 
under transnational and rotational changes are used for the robust description of shape of the boundary. Comer points are 
usually detected in a curvature function space by capturing the points whose curvature values are above a certain threshold 
value. We developed a new comer detection algorithm which provides reliable and invariant comers for a matching procedure 
in the early study[9], A graph can be constructed for a model object using comer points as nodes of the graph. Each node 
has a local feature as well as relational features with other nodes. For the matching process, a similar graph is constructed 
for the input image which may consist of one or several overlapped objects. Each model graph is then matched against the 
input image graph to find the best matching subgraph. 


2.2 Hopfield Network versus Mean field theory 


The continuous Hopfield network(CHN) is a deterministic model which retains the significant characteristics of the 
discrete Hopfield network. The discrete network(DHN) uses binary states. However, real neurons and real physical circuits 
have integrative time delays due to capacitance, and the time evolution of the state of such systems is represented by a 
differential equation, so called the equation of motion. The continuous network has flow of neuron states in a continuous 
domain while the discrete network has flow of neuron states in a discrete domain. It means that CHN is better than DHN for 
the optimization problems since CHN has a smooth energy function surface. A two dimensional array is constructed to apply 
a matching problem into the neural networks. The columns of the array label the nodes of an object model, and the rows 
indicate the nodes of an input object[14,15]. The number of column n is the number of nodes of a model object and the 
number of rows m is the number of nodes of input image. Therefore, the state of each neuron represents the measure of match 
between two nodes from each graph. The matching process can be characterized as minimizing the following energy function: 
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where V* is a variable which converges to *V if the ith node in the input image matches the £th node in the object model; 
otherwise, it converges to "0". The first term in Eq.(l) is a compatibility constraint. Local and relational feature which have 
different measures are normalized to give tolerance for ambiguity of the features as follows: 

c* = K X m ,/*) + W 2 xFV Jt f) + W,x F(r y , r u ) (2) 


The fuzzy function F has a value 1 for a positive support and -1 for a negative support. The value of F(x, y) is defined such 
that if the absolute value of the difference between x and y is less than a threshold 6 , then F(x, y ) is set to 1, otherwise F(x, 
y) is set to -1. The sum of the weights is 1. In our early work[], we uses two features such as angle and distance. Angle 
helps us to recognize the shape of object. However, false segmentation cause to generate different angles from those of 
original segmentation. In this paper, relational features are more emphasized than local features. AHN even works well 
without local features. The last two terms are included to enforce the uniqueness constraint so that each node in the object 
model eventually matches only one node in the input image and the summation of the outputs of the neurons in each row or 
column is no more than 1. Some papers concerning a matching problem with the Hopfield style neural network have used 
EEfi-VJ 2 as an uniqueness constraint. This term implies global restriction. However, matching of occluded objects will not 
guarantee that every row or every column has only one active neuron. Thus the energy function of the occluded matching 
problem excludes the global restriction condition in Eq.(l). In a traveling salesman problem, uniqueness coefficient q is more 
weighted than the coefficient of the compatibility term because q contributes yielding valid solutions. However, conditions 
of valid solutions in the matching of occluded objects are indefinite, so the coefficient A is supposed to be more weighted in 
the matching problem. Eq.(l) can be cast into the discrete Hopfield energy function(DHN) as follows: 
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where =1 when i = j, otherwise 5^=0. Hopfield proved that the energy function is a Liapnov function. Thus the energy 
function converges to a local minimum when the states of neurons converge to stable states[13]. Unlike the other 
application [14, 15], the constraint that LEV* is equal to the number of column can not be used in the occluded object matching 
problem since occluded objects can lose a lot of segments of the original. The matching process of CHN can be characterized 
by the same energy function as that of DHN. Only an integral term is added to the energy function as follows: 

LEW f g-'cwv (4) 
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Where g is a sigmoid function and R A is the input resistance of a neuron. This term comes from the point of view that neural 
input state u & will lag because of the existence of capacitance in an analog electrical circuit. Thus, there is a resistance- 
capacitance charging equation, called the equation of motion that determines the rate of change of ^[14]. It is the first order 
differential equation. *The equation of the motion is as follows[9]: 
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Now, let us consider MFA application. A motion equation is shown in Eq.(5) with the sigmoid function g. Our 
energy function of the matching problem is organized as Eq.(l). The output of each neuron for the matching problem has 
the value of 0 or 1 to represent measure of similarity. We will call output of each neuron a spin for the mean field annealing 
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approach. It was assumed that the spin interactions are symmetric and have no self-interaction (i.e.,7^ = 0). The state 
space of each spin is: , . _ 

s A E {0, 1} for l £ i f k N (7) 


where N 2 = m x n. In simulated annealing, random perturbations move the system towards its thermal equilibrium at the 
current temperature. Assu min g that all the spins are at equilibrium, one can determine the equilibrium spin average of the ith 
spin <s ik > from the Boltzmann distribution and the change in the average system energy as flips from 0 to 1 . To illustrate, 
let H 0 = < H(s) > | M-o* Hj = < H(s) > \ s^.j. Since the system is Boltzmann distributed, the equilibrium value of <s ik > 
is calculated as follows: 
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We define % to represent the quantity H 0 - H„ which is the mean or effective field experienced by the ik th spin. 
Unfortunately, it is in general difficult to compute for large N: 
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The difficulty arises from the fact that and Sj, are not independent, so that their expected values are not separable in the 
above equation. However, when the number of interacting spins is large enough that the effect of any single spin on any other 
spin is very small in comparison to the total field, then the mean field approximation can be used: 
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The Eq.(8) and Eq.(10) has the same structure as Eq.(3) and Eq.(6). In addition, random perturbation to move the system 
towards its thermal equilibrium in simulated annealing is the same as updating rule of the Hopfield network. The only 
difference is that X in eq. (6) is replaced with temperature T. It means that given J, flow to thermal equilibrium in MFA 
is the same as the flow of Hopfield network given X. Therefore, if we find the stable points of states by slowly lowering X 
from the high value, then we will find global solutions or near global solution of the network without initial restriction. We 
call this algorithm as the annealed Hopfield network(AHN). 
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2.3 The Critical Temperature(r c ) and Uniqueness Coefficient^) 


Setting the operating parameters for the annealed network significantly affects a final solution. Starting at too high 
a temperature above the critical temperature is just time-consuming since no progress is made toward a solution until the 
critical temperature is reached. Starting at too low a temperature can quench the system and quickly force it into a poor 
solution. In addition, neural networks often enforce problem constraints through penalty functions which must be weighted 
in importance against the remaining cost components of the objective function. Weighting penalties too heavily leads to valid 
but poor solutions, while reducing the penalties permits infeasible solutions to arise. In this section, techniques for estimating 
T c and q are explained. D. E. Van Den Bout et al. use normalization technique to improve solutions of TSP[16]. However, 
the technique is hard to implement hardware since it is not natural flow in biological neural networks or the analog Hopfield 


network model. In this paper, we use the sigmoid function and derive critical temperature from the function. The spin 
perturbations near T c are small^enough so that all the spins remain near their high temperature average of IfN. With this 
assumption, the effect mean field changes have on the spins are found from the sigmoid function in Eq.(8) to be 
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From Eq.(5), 
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The change of s lk , As-j, cause the change of inputs of the other neurons A u Jt as follows: 
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Now, the change of s jU As u from the change of u jt is: 
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From the change of s Jt , As)„ the new input of ikt h neuron A is calculated: 
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Finally, we get the new perturbation A s£ from the Eq.(15) 
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In fact, the Hopfield network in the object matching problem is a fully connected network and the flow of the output 
change of neurons are very complicated. The result is based on the assumption that output changes of all the other neurons 
caused by the change of the ikth neuron A% are fed back to the ikth neuron and force the change of the ikth neuron to be 
accelerated when a temperature is near Tc. Therefore, we ignore the effect of other neuron outputs to simplify this problem. 
Let an average of connection strength be w. At the critical temperature, the spin perturbation must persist so that Ar^ = A %■ 
This results in: 
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for N > > 1. q is not emphasized because valid solution is not quite definite in this problem. Therefore q has relatively small 


value as it does in the TSP problem, q is set to the unit value in the energy function. 


3. EXPERIMENTAL RESULTS 

Several models are obtained and used in the matching procedure to test the new algorithm. Figure- 1 shows images 
of model objects and occluded images. Once images are obtained, boundary is extracted by the chain code method. After 
extracting the boundary, comer points are obtained by using the optimal boundary smoothing method based on the constrained 
regularization technique. Figure-2 shows the boundary and comer points of the model objects. The number of segments of 
the models and occluded images ranges from 6 to 25. From each segment, features are extracted: an angle as local feature 
and the distance between nodes as relational feature. The boundary segmentation algorithm is very reliable in the sense that 
it is not noise dependable and it keeps detecting the same comer points from an object in different scenes. However, some 
models in occluded images are occasionally oversegmented or lose some comer points under the same threshold value. They 
affect matching procedure as occluded parts does. A matching algorithm should be tolerable for the false segmentation 
occurred in preprocessing stage as well as occlusion. AHN shows good performance in the above situation. Fig.-2(c) and 
pjg -3(d) shows robustness of the algorithm under over-segmentation as well as occlusion. The number of the model is 8 but 
in the occluded image, the model has 14 segments. 8 nodes of the model are exactly matched with those of images. Figure-3 
shows output plots of AHN. Star signs indicate matched nodes between model and occluded images. The results show the 
desired matchings are successfully obtained. 

We also experiment on the critical temperature to see if the parameter estimation is correct. As shown in Figiire-5, 
annealing through the higher temperature is wasted work since it has no effect on the energy function. Instead, most of the 
optimization occurs near the critical temperature. There is a precipitous drop in the energy function around the critical 
temperature where a coagulation starts. Annealing through the low temperature does not improve the solution but serves only 
to saturate the neurons at 1 or 0. As shown in Figure-4, the experiment result of Figure-3(a) was 0.45 of T c while the estimate 
of Tc was 0.7. This discrepancy may result from the small number of neurons of the example since we assume that the number 
of neurons are very large for mean field approximation. Therefore we have the T c be the half of the estimated T c to get a valid 
solution. 


4. CONCLUSION 

Issues related to the reliable matching have been discussed in this paper. Annealing the network allowed convergence 
to begin close to the critical temperature such that good solutions were found. By estimating the critical temperature, we can 
get a near optimal solution by few steps decreasing temperature. In conclusion, AHN gives a reliable matching of the 
corresponding segments between two objects. The method eliminates possibility for a part of an object to be matched to 
similar segments in a different object. We conclude that AHN is a robust approach to solve the two-dimensional occlusion 
problems. 
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